Neural mixture models with expectation-maximization for end-to-end deep clustering

Any clustering algorithm must synchronously learn to model the clusters and allocate data those in absence of labels. Mixture model-based methods with pre-defined statistical distributions based on cluster likelihoods. They iteratively refine distribution parameters member assignments following Expectation-Maximization (EM) algorithm. However, representability such hand-designed that employ a limited amount is not adequate for most real-world tasks. In this paper, we realize mixture neural network where final layer neurons, aid an additional transformation, approximate outputs. The pose as distributions. result elegant, much-generalized representation than restricted We train end-to-end via batch-wise EM iterations forward pass acts E-step backward M-step. image clustering, mixture-based objective can be used along existing learning methods. particular, show when mixture-EM optimization fused consistency optimization, it improves sole performance clustering. Our trained networks outperform single-stage deep still depend k-means, unsupervised classification accuracy 63.8% STL10, 58% CIFAR10, 25.9% CIFAR100, 98.9% MNIST.